The 2002 quality framework in England

The Quality Assurance Agency (QAA), the body responsible for the assessment of the quality of higher education in the UK, introduced a new system of institutional audit in England from 2002-03. This page gives an overview of the process and an assessment of the implications for law schools.

The information is taken from Debbie Lockton’s article ‘The new quality framework and the implications for law schools’ in the The Law Teacher 2002 36(2):212-219. Debbie is Head of Quality for the Faculty of Business and Law at De Montfort University, and one of only four institutional auditors from law.

---

The Quality Assurance Agency’s institutional audit process, which replaces universal subject review, places a much greater reliance on an institution’s in-house quality assurance arrangements, and includes a requirement to publish comprehensive information for students and employers.

The full procedures for England are laid out in the QAA’s
Handbook for institutional audit. Each institution will engage in an institutional audit within a six year cycle. All higher education institutions in England will have been audited by the end of 2005.

The audit will focus on three main areas:

1. the effectiveness of the institution’s internal quality assurance structures and mechanisms
2. the accuracy, completeness and reliability of information, including programme specifications
3. examples of the institution’s quality assurance systems at work at the programme level (discipline audit trails) or across the institution as a whole (thematic enquiries)

Audit teams will consist of 4-6 institutional auditors and an audit secretary. The result of the audit will be the publication of a report, giving a confidence statement in the likely future security of the quality of an institution’s programmes and the academic standards of its awards, plus a statement on the accuracy, integrity, completeness and frankness of published information. The report will make recommendations for further consideration and may also identify areas where the team considers there is good reason for a full review at subject or discipline level.

Discipline audit trails

Discipline audit trails are generally expected to represent some 10% of the institution’s programmes, measured by student FTEs, and are described as a “way of sampling what actually happens at the point where academic and quality standards are determined and achieved”.

Disciplines may be selected in a number of ways:

• the self-evaluation document (SED) submitted to the audit team is unclear about particular parts of the quality assurance arrangements at either institutional or discipline level
• the discipline offers interesting or innovative features
• the documentation indicates a possible or identified weakness at either institutional or discipline level (for example in a past external quality assurance report)
• the discipline offers a recent illustration at discipline level of institutional processes
• the discipline has been identified at the student briefing meeting as raising issues worthy of more detailed examination

In addition, there will be random selection to give a team the opportunity to see typical practice.

Discipline audit trails will normally involve two auditors, at least one of whom will have an understanding of the academic area involved. The trail will consist of a short self-evaluation document (SED) available before the briefing meeting (for example a recent internal review report), provision of a limited amount of documentation, discussions with staff and students about the way the institution’s quality assurance processes are implemented and their effectiveness, discussions with staff and students about the completeness, accuracy and usefulness of published information and discussion of the quality and standards of learning and teaching, which could include discussion with external examiners and external participants in internal reviews.

The auditors will then form judgements about the extent to which the institution’s quality assurance arrangements are in practice operating at the level of programme delivery.

Thematic enquiries

If an audit team feels there is an aspect of the institution’s processes which is particularly interesting, or which needs to be checked across a number of disciplines, it may undertake a thematic enquiry. Evidence for thematic enquiries may come through discipline audit trails. For example, a discipline audit trail in law could also be used to gather information on the use of reports from professional bodies or external examiner reports, or adherence to certain parts of the QAA’s
Code of practice.

Requirements for published information

Institutions will be required to collate and publish much more data than at present. The
information requirements institutions will be required to publish includes:

• a summary of external examiner reports for each programme together with an institution-wide commentary on the findings of external examiners
• student views on the quality of programmes, staff and facilities together with the institution’s own student feedback survey
• summary of the institution’s learning and teaching strategy
• summary statements of internal periodic review and actions taken
• summaries of links with employers, stating how the employer’s needs and views are used to develop programmes
• student entry qualifications
• drop out, progression and completion rates
• data on the class of first degress by subject area
• first destination information for full time first degree students

External verification of standards is crucial in the new process, and the views of students feature much more starkly. All law schools will need to ensure that evidence is available to support claims made by the institution or the discipline in the self-evaluation document. Reviewers and auditors will expect to see evidence going back some time, and law schools should ensure that the evidence base is readily available.

Implications for law schools

According to Debbie Lockton it is likely that in the period 2002-05 the majority of law schools can expect some engagement with the QAA.

Law may well be selected by the institution for a discipline audit trail or thematic enquiry, as it was reviewed some time ago (1993-94). The transitional arrangements which are in place for those institutions awaiting an audit during the period 2002-05 will involve a large number in a developmental engagement with the QAA, and for some it will involve a limited amount of subject review.

Developmental engagements are aimed at providing an opportunity for institutions to test the strength of their internal review procedures at the level of the discipline or programme. They will employ a standardised method, derived from the principles and methods which to be adopted in discipline audit trails. Discipline areas will be agreed after discussion with the institution. There will be an SED, and the engagement will consist of a two day visit, involving discussions with staff and students and scrutinising of documentation. The team will produce a short report, expressing two threshold judgements about the academic standards set and achieved and the quality of the learning opportunities. This report will not be published. Should a departmental engagement identify issues for concern, a full subject review will normally be proposed.

In addition, the Law Society and the Bar Council have put forward objections to the proposal to abolish universal subject review, and the Bar Council has stated that it will consider developing its own procedure for maintaining standards, with any costs incurred to be met by the institutions. Institutions may wish to put forward law as a discipline for either a subject review, developmental engagement or discipline audit trail as a way of meeting the concerns of the professional bodies.

Last Modified: 4 June 2010